A Multi-Level Method for the Steady State Solution of Discrete-Time Markov Chains

Markov chains are one of the most important kinds of models in Simulation. A fast iterative algorithm for the steady state solution of continuous-time Markov chains (CTMCs) was introduced by Horton and Leutenegger [HL94]. The so-called multi-level algorithm utilizes ideas from algebraic multigrid to provide an efficient alternative to the currently used Gauss-Seidel and successive overrelaxation (SOR) methods. This paper examines the applicability of the algorithm and its ideas to discretetime Markov chains (DTMC). The steady state solution is determined by successively coarsening an initial problem and solving it in parallel on various levels of detail. Since CTMC and DTMC are closely related and stiff problems pose the same difficulties for the iterative solution methods Power (DTMC) and SOR (CTMC) the effect of the multi grid approach is equally good. An adapted algorithm is presented and tested on different chains, that were derived from the ones used in testing the original Multi-Level Algorithm. The experiments show, that the runtime can be greatly reduced when using the Power method and a slightly modified aggregation scheme. When using the GaussSeidel method as smoothing algorithm the performance is even better.